Context: The Aarhus University Signal Processing group, in collaboration with University of Southern Denmark, hasrecently released a dataset containing images of unique plants belonging to 12 species at several growth stages. Can a weed be differentiated from a crop seedling?T he ability to do so effectively can mean better crop yields and better stewardship of the environment. The goal of the project is to create a classifier capable of determining a plant's species from a photo.
Data Description:
The data consists of Image file and label csv for 4750 records
The goal of the project is to create a classifier capable of determining a plant's species from a photo
#Reading the training images from the path and labelling them into the given categories
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import cv2
#import os
import seaborn as sns # for data visualization
import tensorflow as tf
import keras
from tensorflow.keras.models import Sequential #sequential api for sequential model
from tensorflow.keras.layers import Dense, Dropout, Flatten #importing different layers
from tensorflow.keras.layers import Conv2D, MaxPooling2D, BatchNormalization, Activation, Input, LeakyReLU,Activation
from tensorflow.keras import backend as K
from tensorflow.keras.utils import to_categorical #to perform one-hot encoding
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D
#from keras.optimizers import RMSprop,Adam #optimiers for optimizing the model
from keras.callbacks import EarlyStopping #regularization method to prevent the overfitting
from keras.callbacks import ModelCheckpoint
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras import losses, optimizers
from tensorflow.keras.optimizers import RMSprop,Adam
# loading Images and labels
# Separating the images and labels
X_train = []
y_train = []
y_train = pd.read_csv("D:\\course\\Labels.CSV")
X_train = np.load('D:\\course\\Images.npy')
X_train.shape
(4750, 128, 128, 3)
y_train.describe()
| Label | |
|---|---|
| count | 4750 |
| unique | 12 |
| top | Loose Silky-bent |
| freq | 654 |
# Distinct values of seedlings
y_train['Label'].unique()
array(['Small-flowered Cranesbill', 'Fat Hen', 'Shepherds Purse',
'Common wheat', 'Common Chickweed', 'Charlock', 'Cleavers',
'Scentless Mayweed', 'Sugar beet', 'Maize', 'Black-grass',
'Loose Silky-bent'], dtype=object)
CATEGORIES = ["Small-flowered Cranesbill", "Fat Hen", "Shepherds Purse",
"Common wheat", "Common Chickweed", "Charlock", "Cleavers",
"Scentless Mayweed", "Sugar beet", "Maize", "Black-grass",
"Loose Silky-bent"]
df = y_train
smallfcrimg = []
fathenimg = []
spimg = []
cwimg = []
ccimg = []
charlockimg = []
climg = []
smimg = []
sbimg = []
maizeimg = []
bgimg = []
lsbimg = []
for i in range(0,4750):
if (df.Label[i] == 'Small-flowered Cranesbill'):
smallfcrimg.append(X_train[i])
elif (df.Label[i] == 'Fat Hen'):
fathenimg.append(X_train[i])
elif (df.Label[i] == 'Shepherds Purse'):
spimg.append(X_train[i])
elif (df.Label[i] == 'Common wheat'):
cwimg.append(X_train[i])
elif (df.Label[i] == 'Common Chickweed'):
ccimg.append(X_train[i])
elif (df.Label[i] == 'Charlock'):
charlockimg.append(X_train[i])
elif (df.Label[i] == 'Cleavers'):
climg.append(X_train[i])
elif (df.Label[i] == 'Scentless Mayweed'):
smimg.append(X_train[i])
elif (df.Label[i] == 'Sugar beet'):
sbimg.append(X_train[i])
elif (df.Label[i] == 'Maize'):
maizeimg.append(X_train[i])
elif (df.Label[i] == 'Black-grass'):
bgimg.append(X_train[i])
elif (df.Label[i] == 'Loose Silky-bent'):
lsbimg.append(X_train[i])
from keras.preprocessing import image
# plotting 2 x 3 image matrix
fig = plt.figure(figsize = (30,60))
for i in range(36):
if i < 3:
fp = smallfcrimg[i]
label = 'Small-flowered Cranesbill'
if i>=3 and i<6:
fp = fathenimg[i-3]
label = 'Fat Hen'
if i>=6 and i<9:
fp = spimg[i-6]
label = 'Shepherds Purse'
if i>=9 and i<12:
fp = cwimg[i-9]
label = 'Common wheat'
if i>=12 and i<15:
fp = ccimg[i-12]
label = 'Common Chickweed'
if i>=15 and i<18:
fp = charlockimg[i-15]
label = 'Charlock'
if i>=18 and i<21:
fp = climg[i-18]
label = 'Cleavers'
if i>=21 and i<24:
fp = smimg[i-21]
label = 'Scentless Mayweed'
if i>=24 and i<27:
fp = sbimg[i-24]
label = 'Sugar beet'
if i>=27 and i<30:
fp = maizeimg[i-27]
label = 'Maize'
if i>=30 and i<33:
fp = bgimg[i-30]
label = 'Black-grass'
if i>=33 and i<36:
fp = lsbimg[i-33]
label = 'Loose Silky-bent'
#ax = fig.add_subplot(1, 2, i+1)
# to plot without rescaling, remove target_size
plt.subplot(12,3,i+1)
plt.imshow(fp)
plt.title(label)
plt.axis('off')
plt.show()
# Normalizing pixel values
X_train = X_train/255.0
df = y_train
print((df.Label.values == 'Small-flowered Cranesbill').argmax())
print((df.Label.values == 'Fat Hen').argmax())
0 496
#plotting the pie chart
labels = 'Small-flowered Cranesbill', 'Fat Hen', 'Shepherds Purse','Common wheat', 'Common Chickweed', 'Charlock', 'Cleavers','Scentless Mayweed', 'Sugar beet', 'Maize', 'Black-grass','Loose Silky-bent'
sizes = [df.Label[df['Label']=='Small-flowered Cranesbill'].count(), df.Label[df['Label']=='Fat Hen'].count(),
df.Label[df['Label']=='Shepherds Purse'].count(), df.Label[df['Label']=='Common wheat'].count(),
df.Label[df['Label']=='Common Chickweed'].count(), df.Label[df['Label']=='Charlock'].count(),
df.Label[df['Label']=='Cleavers'].count(), df.Label[df['Label']=='Scentless Mayweed'].count(),
df.Label[df['Label']=='Sugar beet'].count(), df.Label[df['Label']=='Maize'].count(),
df.Label[df['Label']=='Black-grass'].count(), df.Label[df['Label']=='Loose Silky-bent'].count()
]
#explode = (0, 0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0,1.1)
explode = (0, 0,0,0,0,0,0,0,0,0,0,0)
fig1, ax1 = plt.subplots(figsize=(10, 8))
ax1.pie(sizes, explode=explode, labels=labels, autopct='%1.1f%%',
shadow=True, startangle=90)
ax1.axis('equal')
plt.title("Proportion of Seedlings in the sample ", size = 20)
plt.show()
The above plot shows that the distribution of seedlings in the dataset is imbalanced,
X_train_new = []
sets = []; getEx = True
for i in X_train:
blurr = cv2.GaussianBlur(i,(5,5),0)
X_train_new.append(blurr)
# hsv = cv2.cvtColor(blurr,cv2.COLOR_BGRA2RGB)
#GREEN PARAMETERS
lower = (25,40,50)
upper = (75,255,255)
# mask = cv2.inRange(hsv,lower,upper)
struc = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(11,11))
# mask = cv2.morphologyEx(mask,cv2.MORPH_CLOSE,struc)
# boolean = mask>0
new = np.zeros_like(i,np.uint8)
# new[boolean] = i[boolean]
# new_train.append(new)
X_train_new = np.asarray(X_train_new)
# CLEANED IMAGES
for i in range(1):
plt.imshow(X_train_new[i])
smallfcrimgnew = []
fathenimgnew = []
spimgnew = []
cwimgnew = []
ccimgnew = []
charlockimgnew = []
climgnew = []
smimgnew = []
sbimgnew = []
maizeimgnew = []
bgimgnew = []
lsbimgnew = []
for i in range(0,4750):
if (df.Label[i] == 'Small-flowered Cranesbill'):
smallfcrimgnew.append(X_train_new[i])
elif (df.Label[i] == 'Fat Hen'):
fathenimgnew.append(X_train_new[i])
elif (df.Label[i] == 'Shepherds Purse'):
spimgnew.append(X_train_new[i])
elif (df.Label[i] == 'Common wheat'):
cwimgnew.append(X_train_new[i])
elif (df.Label[i] == 'Common Chickweed'):
ccimgnew.append(X_train_new[i])
elif (df.Label[i] == 'Charlock'):
charlockimgnew.append(X_train_new[i])
elif (df.Label[i] == 'Cleavers'):
climgnew.append(X_train_new[i])
elif (df.Label[i] == 'Scentless Mayweed'):
smimgnew.append(X_train_new[i])
elif (df.Label[i] == 'Sugar beet'):
sbimgnew.append(X_train_new[i])
elif (df.Label[i] == 'Maize'):
maizeimgnew.append(X_train_new[i])
elif (df.Label[i] == 'Black-grass'):
bgimgnew.append(X_train_new[i])
elif (df.Label[i] == 'Loose Silky-bent'):
lsbimgnew.append(X_train_new[i])
from keras.preprocessing import image
# plotting 2 x 3 image matrix
fig = plt.figure(figsize = (30,60))
for i in range(36):
if i < 3:
fp = smallfcrimgnew[i]
label = 'Small-flowered Cranesbill'
if i>=3 and i<6:
fp = fathenimgnew[i-3]
label = 'Fat Hen'
if i>=6 and i<9:
fp = spimgnew[i-6]
label = 'Shepherds Purse'
if i>=9 and i<12:
fp = cwimgnew[i-9]
label = 'Common wheat'
if i>=12 and i<15:
fp = ccimgnew[i-12]
label = 'Common Chickweed'
if i>=15 and i<18:
fp = charlockimgnew[i-15]
label = 'Charlock'
if i>=18 and i<21:
fp = climgnew[i-18]
label = 'Cleavers'
if i>=21 and i<24:
fp = smimgnew[i-21]
label = 'Scentless Mayweed'
if i>=24 and i<27:
fp = sbimgnew[i-24]
label = 'Sugar beet'
if i>=27 and i<30:
fp = maizeimgnew[i-27]
label = 'Maize'
if i>=30 and i<33:
fp = bgimgnew[i-30]
label = 'Black-grass'
if i>=33 and i<36:
fp = lsbimgnew[i-33]
label = 'Loose Silky-bent'
#ax = fig.add_subplot(1, 2, i+1)
# to plot without rescaling, remove target_size
plt.subplot(12,3,i+1)
plt.imshow(fp)
plt.title(label)
plt.axis('off')
plt.show()
Finding the mean images for each class of tumor:
def find_mean_img(full_mat, title):
# calculate the average
mean_img = np.mean(full_mat, axis = 0)
plt.imshow(mean_img)
plt.title(f'Average {title}')
plt.axis('off')
plt.show()
return mean_img
sfc_mean = find_mean_img(np.array(smallfcrimgnew), 'Small-flowered Cranesbill')
fh_mean = find_mean_img(np.array(fathenimgnew), 'Fat Hen')
sp_mean = find_mean_img(np.array(spimgnew), 'Shepherd Purse')
cw_mean = find_mean_img(np.array(cwimgnew), 'Common Weed')
cc_mean = find_mean_img(np.array(ccimgnew), 'Common chickweed')
cc_mean = find_mean_img(np.array(ccimgnew), 'Common chickweed')
charlock_mean = find_mean_img(np.array(charlockimgnew), 'Charlock')
sm_mean = find_mean_img(np.array(smimgnew), 'Scentless mayweed')
sb_mean = find_mean_img(np.array(sbimgnew), 'Sugar beet')
maize_mean = find_mean_img(np.array(maizeimgnew), 'Maize')
bg_mean = find_mean_img(np.array(bgimgnew), 'Black-grass')
lsb_mean = find_mean_img(np.array(lsbimgnew), 'Loose Silky-bent')
from sklearn import preprocessing
from keras.utils import np_utils
from sklearn.model_selection import train_test_split
y_train['Label'].unique()
array(['Small-flowered Cranesbill', 'Fat Hen', 'Shepherds Purse',
'Common wheat', 'Common Chickweed', 'Charlock', 'Cleavers',
'Scentless Mayweed', 'Sugar beet', 'Maize', 'Black-grass',
'Loose Silky-bent'], dtype=object)
x_train,x_test,y_train,y_test = train_test_split(X_train_new,y_train,test_size=0.3,random_state=7,stratify=y_train)
y_train_n = np.asarray(y_train)
y_train_n= y_train_n.reshape((len(y_train_n), ))
le = preprocessing.LabelEncoder()
le.fit(y_train_n)
y_train_enc = le.transform(y_train_n)
y_test_n = np.asarray(y_test)
y_test_n= y_test_n.reshape((len(y_test_n), ))
le = preprocessing.LabelEncoder()
le.fit(y_test_n)
y_test_enc = le.transform(y_test_n)
y_train_c = to_categorical(y_train_enc, 12)
y_test_c = to_categorical(y_test_enc, 12)
y_train_o = to_categorical(y_train_enc)
y_test_o = to_categorical(y_test_enc)
y_train_o = to_categorical(y_train_enc)
print(y_train_c.shape)
print(y_test_c.shape)
print(x_train.shape)
print(y_train_o.shape)
print(y_test_o.shape)
(3325, 12) (1425, 12) (3325, 128, 128, 3) (3325, 12) (1425, 12)
print(y_train_enc[223])
6
y_train_r = y_train_enc.reshape((len(y_train_enc), 1, 1))
y_test_r = y_test_enc.reshape((len(y_test_enc), 1, 1))
We will be using convoluation neural netwrok for model building:
Model 1: CNN with Dropout
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=20)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
model = Sequential()
#
y_train=np.array(y_train)
model.add(Conv2D(filters = 64, kernel_size = (5,5),padding = 'Same',
activation ='relu', input_shape = (128,128,3)))
model.add(MaxPool2D(pool_size=(2,2)))
model.add(Dropout(0.25))
#
model.add(Conv2D(filters = 128, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model.add(Dropout(0.25))
#
model.add(Conv2D(filters = 128, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model.add(Dropout(0.3))
#
model.add(Conv2D(filters = 128, kernel_size = (2,2),padding = 'Same',
activation ='relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model.add(Dropout(0.3))
#
model.add(Conv2D(filters = 256, kernel_size = (2,2),padding = 'Same',
activation ='relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model.add(Dropout(0.3))
#
model.add(Flatten())
model.add(Dense(1024, activation = "relu"))
model.add(Dropout(0.5))
model.add(Dense(12, activation = "softmax"))
optimizer = Adam(lr=0.001, beta_1=0.9, beta_2=0.999)
model.compile(optimizer = optimizer , loss = "categorical_crossentropy", metrics=["accuracy"])
epochs = 200
batch_size = 64
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=5)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
history=model.fit(x_train,
y_train_c, #It expects integers because of the sparse_categorical_crossentropy loss function
epochs=30, #number of iterations over the entire dataset to train on
batch_size=64,validation_split=0.20,callbacks=[es, mc],use_multiprocessing=True)#number of samples per gradient update for training
C:\Users\appradha.ORADEV\AppData\Roaming\Python\Python37\site-packages\keras\optimizer_v2\optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead. "The `lr` argument is deprecated, use `learning_rate` instead.")
Epoch 1/30 42/42 [==============================] - 163s 4s/step - loss: 2.4433 - accuracy: 0.1335 - val_loss: 2.4422 - val_accuracy: 0.1338 Epoch 00001: val_accuracy improved from -inf to 0.13383, saving model to best_model.h5 Epoch 2/30 42/42 [==============================] - 175s 4s/step - loss: 2.4252 - accuracy: 0.1256 - val_loss: 2.4363 - val_accuracy: 0.1338 Epoch 00002: val_accuracy did not improve from 0.13383 Epoch 3/30 42/42 [==============================] - 177s 4s/step - loss: 2.4028 - accuracy: 0.1553 - val_loss: 2.2685 - val_accuracy: 0.2391 Epoch 00003: val_accuracy improved from 0.13383 to 0.23910, saving model to best_model.h5 Epoch 4/30 42/42 [==============================] - 175s 4s/step - loss: 2.0032 - accuracy: 0.3263 - val_loss: 1.8115 - val_accuracy: 0.3624 Epoch 00004: val_accuracy improved from 0.23910 to 0.36241, saving model to best_model.h5 Epoch 5/30 42/42 [==============================] - 184s 4s/step - loss: 1.7065 - accuracy: 0.3992 - val_loss: 1.6021 - val_accuracy: 0.4797 Epoch 00005: val_accuracy improved from 0.36241 to 0.47970, saving model to best_model.h5 Epoch 6/30 42/42 [==============================] - 181s 4s/step - loss: 1.5374 - accuracy: 0.4511 - val_loss: 1.5008 - val_accuracy: 0.4812 Epoch 00006: val_accuracy improved from 0.47970 to 0.48120, saving model to best_model.h5 Epoch 7/30 42/42 [==============================] - 175s 4s/step - loss: 1.3415 - accuracy: 0.5286 - val_loss: 1.2945 - val_accuracy: 0.5940 Epoch 00007: val_accuracy improved from 0.48120 to 0.59398, saving model to best_model.h5 Epoch 8/30 42/42 [==============================] - 206s 5s/step - loss: 1.2490 - accuracy: 0.5692 - val_loss: 1.5495 - val_accuracy: 0.4571 Epoch 00008: val_accuracy did not improve from 0.59398 Epoch 9/30 42/42 [==============================] - 206s 5s/step - loss: 1.1297 - accuracy: 0.5917 - val_loss: 1.1583 - val_accuracy: 0.6271 Epoch 00009: val_accuracy improved from 0.59398 to 0.62707, saving model to best_model.h5 Epoch 10/30 42/42 [==============================] - 213s 5s/step - loss: 1.0462 - accuracy: 0.6293 - val_loss: 1.2997 - val_accuracy: 0.5323 Epoch 00010: val_accuracy did not improve from 0.62707 Epoch 11/30 42/42 [==============================] - 201s 5s/step - loss: 0.9756 - accuracy: 0.6515 - val_loss: 1.1571 - val_accuracy: 0.5805 Epoch 00011: val_accuracy did not improve from 0.62707 Epoch 12/30 42/42 [==============================] - 193s 5s/step - loss: 0.8792 - accuracy: 0.6883 - val_loss: 1.0429 - val_accuracy: 0.6541 Epoch 00012: val_accuracy improved from 0.62707 to 0.65414, saving model to best_model.h5 Epoch 13/30 42/42 [==============================] - 183s 4s/step - loss: 0.8094 - accuracy: 0.7124 - val_loss: 0.9389 - val_accuracy: 0.6932 Epoch 00013: val_accuracy improved from 0.65414 to 0.69323, saving model to best_model.h5 Epoch 14/30 42/42 [==============================] - 182s 4s/step - loss: 0.7645 - accuracy: 0.7252 - val_loss: 0.8558 - val_accuracy: 0.7248 Epoch 00014: val_accuracy improved from 0.69323 to 0.72481, saving model to best_model.h5 Epoch 15/30 42/42 [==============================] - 179s 4s/step - loss: 0.6920 - accuracy: 0.7571 - val_loss: 0.8112 - val_accuracy: 0.7609 Epoch 00015: val_accuracy improved from 0.72481 to 0.76090, saving model to best_model.h5 Epoch 16/30 42/42 [==============================] - 177s 4s/step - loss: 0.6491 - accuracy: 0.7658 - val_loss: 0.8358 - val_accuracy: 0.7233 Epoch 00016: val_accuracy did not improve from 0.76090 Epoch 17/30 42/42 [==============================] - 202s 5s/step - loss: 0.6100 - accuracy: 0.7786 - val_loss: 0.8072 - val_accuracy: 0.7233 Epoch 00017: val_accuracy did not improve from 0.76090 Epoch 18/30 42/42 [==============================] - 202s 5s/step - loss: 0.5496 - accuracy: 0.8011 - val_loss: 0.8082 - val_accuracy: 0.7308 Epoch 00018: val_accuracy did not improve from 0.76090 Epoch 19/30 42/42 [==============================] - 186s 4s/step - loss: 0.5055 - accuracy: 0.8064 - val_loss: 0.9016 - val_accuracy: 0.7143 Epoch 00019: val_accuracy did not improve from 0.76090 Epoch 20/30 42/42 [==============================] - 177s 4s/step - loss: 0.5071 - accuracy: 0.8229 - val_loss: 0.7579 - val_accuracy: 0.7489 Epoch 00020: val_accuracy did not improve from 0.76090 Epoch 21/30 42/42 [==============================] - 189s 5s/step - loss: 0.4327 - accuracy: 0.8410 - val_loss: 0.7432 - val_accuracy: 0.7504 Epoch 00021: val_accuracy did not improve from 0.76090 Epoch 22/30 42/42 [==============================] - 181s 4s/step - loss: 0.4363 - accuracy: 0.8451 - val_loss: 0.6889 - val_accuracy: 0.7759 Epoch 00022: val_accuracy improved from 0.76090 to 0.77594, saving model to best_model.h5 Epoch 23/30 42/42 [==============================] - 187s 4s/step - loss: 0.4030 - accuracy: 0.8564 - val_loss: 0.7739 - val_accuracy: 0.7549 Epoch 00023: val_accuracy did not improve from 0.77594 Epoch 24/30 42/42 [==============================] - 180s 4s/step - loss: 0.3731 - accuracy: 0.8620 - val_loss: 0.8184 - val_accuracy: 0.7459 Epoch 00024: val_accuracy did not improve from 0.77594 Epoch 25/30 42/42 [==============================] - 179s 4s/step - loss: 0.3752 - accuracy: 0.8508 - val_loss: 0.7284 - val_accuracy: 0.7729 Epoch 00025: val_accuracy did not improve from 0.77594 Epoch 26/30 42/42 [==============================] - 178s 4s/step - loss: 0.3383 - accuracy: 0.8722 - val_loss: 0.7295 - val_accuracy: 0.7684 Epoch 00026: val_accuracy did not improve from 0.77594 Epoch 27/30 42/42 [==============================] - 188s 4s/step - loss: 0.3448 - accuracy: 0.8695 - val_loss: 0.7211 - val_accuracy: 0.7624 Epoch 00027: val_accuracy did not improve from 0.77594 Epoch 00027: early stopping
model.evaluate(x_test,np.array(y_test_c))
45/45 [==============================] - 19s 411ms/step - loss: 0.6089 - accuracy: 0.8147
[0.6089368462562561, 0.8147368431091309]
Model 2: CNN with Dropout after Convolution and having two Dense layers with 16 & 32 units respectively
Since CNN Model 1 gives an ok test accuracy and appears to be slighlty overfitting on the training dataset, let's use CNN Model 2, which has a different architecture that should generalize well and not overfit.
class conv_Layers:
def __init__(self, nfilters, kernel_size, stride=1,
pool_size=2, leakyrelu_slope=0.1, dropc=0.0, bnorm=False):
self.nfilters = nfilters
self.kernel_size = kernel_size
self.stride = stride
self.pool_size = pool_size
self.leakyrelu_slope = leakyrelu_slope
self.dropfrac = dropc
self.bnorm = bnorm
def __call__(self, x):
x = Conv2D(self.nfilters, kernel_size=self.kernel_size,
strides=self.stride, padding='same')(x)
x = LeakyReLU(self.leakyrelu_slope)(x)
if (self.dropfrac > 0.0):
x = Dropout(self.dropfrac)(x)
if (self.bnorm):
x = BatchNormalization()(x)
x = MaxPool2D(self.pool_size)(x)
return x
class dense_Layers:
def __init__(self, nunits, leakyrelu_slope=0.1, dropd=0.0, bnorm=False):
self.nunits = nunits
self.leakyrelu_slope = leakyrelu_slope
self.dropfrac = dropd
self.bnorm = bnorm
def __call__(self, x):
x = Dense(self.nunits)(x)
x = LeakyReLU(self.leakyrelu_slope)(x)
if (self.dropfrac > 0.0):
x = Dropout(self.dropfrac)(x)
if (self.bnorm):
x = BatchNormalization()(x)
return x
def LNmodel(in_shape, conv_filters, dense_filters, kernel_size, num_classes, lr,
stride=1, pool_size=2, leakyrelu_slope=0.1, dropc=0.0, dropd=0.0, bnorm=False):
in_shape = X_train.shape[1:]
i = Input(shape=in_shape)
for ncl, nconvfilters in enumerate(conv_filters):
if (ncl==0):
x = conv_Layers(nconvfilters, kernel_size,
stride, pool_size, leakyrelu_slope, dropc, bnorm)(i)
else:
x = conv_Layers(nconvfilters, kernel_size,
stride, pool_size, leakyrelu_slope, dropc, bnorm)(x)
x = Flatten()(x)
for ndl, ndunits in enumerate(dense_filters):
x = dense_Layers(ndunits, leakyrelu_slope, dropd, bnorm)(x)
x = Dense(num_classes, activation='softmax')(x)
ln_model = Model(inputs=i, outputs=x)
adam = optimizers.Adam(lr=lr)
ln_model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
return ln_model
lr = 0.001
kernelsize = 5
in_shape= X_train.shape[1:]
model_ln3 = LNmodel(in_shape, [16,32], [32,16], kernelsize, 12, lr,
stride=1, pool_size=2, leakyrelu_slope=0.1, dropc=0.30,
dropd=0.5, bnorm=False)
model_ln3.summary()
Model: "model" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) [(None, 128, 128, 3)] 0 _________________________________________________________________ conv2d_5 (Conv2D) (None, 128, 128, 16) 1216 _________________________________________________________________ leaky_re_lu (LeakyReLU) (None, 128, 128, 16) 0 _________________________________________________________________ dropout_6 (Dropout) (None, 128, 128, 16) 0 _________________________________________________________________ max_pooling2d_5 (MaxPooling2 (None, 64, 64, 16) 0 _________________________________________________________________ conv2d_6 (Conv2D) (None, 64, 64, 32) 12832 _________________________________________________________________ leaky_re_lu_1 (LeakyReLU) (None, 64, 64, 32) 0 _________________________________________________________________ dropout_7 (Dropout) (None, 64, 64, 32) 0 _________________________________________________________________ max_pooling2d_6 (MaxPooling2 (None, 32, 32, 32) 0 _________________________________________________________________ flatten_1 (Flatten) (None, 32768) 0 _________________________________________________________________ dense_2 (Dense) (None, 32) 1048608 _________________________________________________________________ leaky_re_lu_2 (LeakyReLU) (None, 32) 0 _________________________________________________________________ dropout_8 (Dropout) (None, 32) 0 _________________________________________________________________ dense_3 (Dense) (None, 16) 528 _________________________________________________________________ leaky_re_lu_3 (LeakyReLU) (None, 16) 0 _________________________________________________________________ dropout_9 (Dropout) (None, 16) 0 _________________________________________________________________ dense_4 (Dense) (None, 12) 204 ================================================================= Total params: 1,063,388 Trainable params: 1,063,388 Non-trainable params: 0 _________________________________________________________________
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=20)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
history_model_ln3 = model_ln3.fit(x_train, y_train_c,
validation_split=0.1,
verbose=1, batch_size=256,
shuffle=True, epochs=60,callbacks=[es,mc])
Epoch 1/60 12/12 [==============================] - 58s 5s/step - loss: 2.4901 - accuracy: 0.1156 - val_loss: 2.4512 - val_accuracy: 0.2222 Epoch 00001: val_accuracy improved from -inf to 0.22222, saving model to best_model.h5 Epoch 2/60 12/12 [==============================] - 60s 5s/step - loss: 2.4064 - accuracy: 0.1648 - val_loss: 2.2999 - val_accuracy: 0.3123 Epoch 00002: val_accuracy improved from 0.22222 to 0.31231, saving model to best_model.h5 Epoch 3/60 12/12 [==============================] - 62s 5s/step - loss: 2.2663 - accuracy: 0.2203 - val_loss: 2.2110 - val_accuracy: 0.3093 Epoch 00003: val_accuracy did not improve from 0.31231 Epoch 4/60 12/12 [==============================] - 66s 6s/step - loss: 2.2024 - accuracy: 0.2477 - val_loss: 2.1432 - val_accuracy: 0.3423 Epoch 00004: val_accuracy improved from 0.31231 to 0.34234, saving model to best_model.h5 Epoch 5/60 12/12 [==============================] - 73s 6s/step - loss: 2.1150 - accuracy: 0.2841 - val_loss: 2.0722 - val_accuracy: 0.3483 Epoch 00005: val_accuracy improved from 0.34234 to 0.34835, saving model to best_model.h5 Epoch 6/60 12/12 [==============================] - 76s 6s/step - loss: 2.0736 - accuracy: 0.3105 - val_loss: 2.0243 - val_accuracy: 0.3483 Epoch 00006: val_accuracy did not improve from 0.34835 Epoch 7/60 12/12 [==============================] - 69s 6s/step - loss: 1.9891 - accuracy: 0.3312 - val_loss: 1.9252 - val_accuracy: 0.3664 Epoch 00007: val_accuracy improved from 0.34835 to 0.36637, saving model to best_model.h5 Epoch 8/60 12/12 [==============================] - 71s 6s/step - loss: 1.9430 - accuracy: 0.3476 - val_loss: 1.9297 - val_accuracy: 0.3604 Epoch 00008: val_accuracy did not improve from 0.36637 Epoch 9/60 12/12 [==============================] - 73s 6s/step - loss: 1.9168 - accuracy: 0.3499 - val_loss: 1.9198 - val_accuracy: 0.3784 Epoch 00009: val_accuracy improved from 0.36637 to 0.37838, saving model to best_model.h5 Epoch 10/60 12/12 [==============================] - 67s 6s/step - loss: 1.8779 - accuracy: 0.3676 - val_loss: 1.9062 - val_accuracy: 0.3994 Epoch 00010: val_accuracy improved from 0.37838 to 0.39940, saving model to best_model.h5 Epoch 11/60 12/12 [==============================] - 65s 5s/step - loss: 1.8466 - accuracy: 0.3757 - val_loss: 1.8266 - val_accuracy: 0.4024 Epoch 00011: val_accuracy improved from 0.39940 to 0.40240, saving model to best_model.h5 Epoch 12/60 12/12 [==============================] - 66s 6s/step - loss: 1.7559 - accuracy: 0.4178 - val_loss: 1.7927 - val_accuracy: 0.4294 Epoch 00012: val_accuracy improved from 0.40240 to 0.42943, saving model to best_model.h5 Epoch 13/60 12/12 [==============================] - 65s 5s/step - loss: 1.7082 - accuracy: 0.4198 - val_loss: 1.8197 - val_accuracy: 0.4114 Epoch 00013: val_accuracy did not improve from 0.42943 Epoch 14/60 12/12 [==============================] - 66s 5s/step - loss: 1.6851 - accuracy: 0.4251 - val_loss: 1.7267 - val_accuracy: 0.4805 Epoch 00014: val_accuracy improved from 0.42943 to 0.48048, saving model to best_model.h5 Epoch 15/60 12/12 [==============================] - 65s 5s/step - loss: 1.6333 - accuracy: 0.4519 - val_loss: 1.7292 - val_accuracy: 0.4685 Epoch 00015: val_accuracy did not improve from 0.48048 Epoch 16/60 12/12 [==============================] - 66s 6s/step - loss: 1.6174 - accuracy: 0.4509 - val_loss: 1.7562 - val_accuracy: 0.4354 Epoch 00016: val_accuracy did not improve from 0.48048 Epoch 17/60 12/12 [==============================] - 66s 5s/step - loss: 1.5548 - accuracy: 0.4823 - val_loss: 1.7492 - val_accuracy: 0.4264 Epoch 00017: val_accuracy did not improve from 0.48048 Epoch 18/60 12/12 [==============================] - 67s 6s/step - loss: 1.5681 - accuracy: 0.4826 - val_loss: 1.7198 - val_accuracy: 0.4234 Epoch 00018: val_accuracy did not improve from 0.48048 Epoch 19/60 12/12 [==============================] - 68s 6s/step - loss: 1.4956 - accuracy: 0.5067 - val_loss: 1.6394 - val_accuracy: 0.4685 Epoch 00019: val_accuracy did not improve from 0.48048 Epoch 20/60 12/12 [==============================] - 65s 5s/step - loss: 1.4575 - accuracy: 0.5070 - val_loss: 1.5683 - val_accuracy: 0.5435 Epoch 00020: val_accuracy improved from 0.48048 to 0.54354, saving model to best_model.h5 Epoch 21/60 12/12 [==============================] - 65s 5s/step - loss: 1.4258 - accuracy: 0.5241 - val_loss: 1.6376 - val_accuracy: 0.4805 Epoch 00021: val_accuracy did not improve from 0.54354 Epoch 22/60 12/12 [==============================] - 66s 5s/step - loss: 1.4159 - accuracy: 0.5267 - val_loss: 1.5935 - val_accuracy: 0.4925 Epoch 00022: val_accuracy did not improve from 0.54354 Epoch 23/60 12/12 [==============================] - 66s 6s/step - loss: 1.3849 - accuracy: 0.5277 - val_loss: 1.5141 - val_accuracy: 0.5285 Epoch 00023: val_accuracy did not improve from 0.54354 Epoch 24/60 12/12 [==============================] - 69s 6s/step - loss: 1.3628 - accuracy: 0.5394 - val_loss: 1.5551 - val_accuracy: 0.4985 Epoch 00024: val_accuracy did not improve from 0.54354 Epoch 25/60 12/12 [==============================] - 66s 6s/step - loss: 1.3157 - accuracy: 0.5592 - val_loss: 1.5369 - val_accuracy: 0.4985 Epoch 00025: val_accuracy did not improve from 0.54354 Epoch 26/60 12/12 [==============================] - 168s 15s/step - loss: 1.2641 - accuracy: 0.5712 - val_loss: 1.4399 - val_accuracy: 0.5556 Epoch 00026: val_accuracy improved from 0.54354 to 0.55556, saving model to best_model.h5 Epoch 27/60 12/12 [==============================] - 89s 7s/step - loss: 1.2323 - accuracy: 0.5755 - val_loss: 1.5271 - val_accuracy: 0.5135 Epoch 00027: val_accuracy did not improve from 0.55556 Epoch 28/60 12/12 [==============================] - 63s 5s/step - loss: 1.2318 - accuracy: 0.5765 - val_loss: 1.5141 - val_accuracy: 0.5015 Epoch 00028: val_accuracy did not improve from 0.55556 Epoch 29/60 12/12 [==============================] - 61s 5s/step - loss: 1.1706 - accuracy: 0.6076 - val_loss: 1.5097 - val_accuracy: 0.5405 Epoch 00029: val_accuracy did not improve from 0.55556 Epoch 30/60 12/12 [==============================] - 63s 5s/step - loss: 1.1807 - accuracy: 0.5896 - val_loss: 1.5333 - val_accuracy: 0.5075 Epoch 00030: val_accuracy did not improve from 0.55556 Epoch 31/60 12/12 [==============================] - 63s 5s/step - loss: 1.1121 - accuracy: 0.6207 - val_loss: 1.5987 - val_accuracy: 0.5165 Epoch 00031: val_accuracy did not improve from 0.55556 Epoch 32/60 12/12 [==============================] - 64s 5s/step - loss: 1.1175 - accuracy: 0.6203 - val_loss: 1.6441 - val_accuracy: 0.4895 Epoch 00032: val_accuracy did not improve from 0.55556 Epoch 33/60 12/12 [==============================] - 64s 5s/step - loss: 1.0858 - accuracy: 0.6193 - val_loss: 1.3927 - val_accuracy: 0.5766 Epoch 00033: val_accuracy improved from 0.55556 to 0.57658, saving model to best_model.h5 Epoch 34/60 12/12 [==============================] - 67s 6s/step - loss: 1.1030 - accuracy: 0.6360 - val_loss: 1.4431 - val_accuracy: 0.5646 Epoch 00034: val_accuracy did not improve from 0.57658 Epoch 35/60 12/12 [==============================] - 64s 5s/step - loss: 1.0483 - accuracy: 0.6357 - val_loss: 1.5012 - val_accuracy: 0.5526 Epoch 00035: val_accuracy did not improve from 0.57658 Epoch 36/60 12/12 [==============================] - 63s 5s/step - loss: 1.0435 - accuracy: 0.6344 - val_loss: 1.4648 - val_accuracy: 0.5586 Epoch 00036: val_accuracy did not improve from 0.57658 Epoch 37/60 12/12 [==============================] - 62s 5s/step - loss: 1.0289 - accuracy: 0.6524 - val_loss: 1.4847 - val_accuracy: 0.5255 Epoch 00037: val_accuracy did not improve from 0.57658 Epoch 38/60 12/12 [==============================] - 62s 5s/step - loss: 0.9854 - accuracy: 0.6544 - val_loss: 1.5154 - val_accuracy: 0.5465 Epoch 00038: val_accuracy did not improve from 0.57658 Epoch 39/60 12/12 [==============================] - 64s 5s/step - loss: 0.9867 - accuracy: 0.6527 - val_loss: 1.5760 - val_accuracy: 0.5255 Epoch 00039: val_accuracy did not improve from 0.57658 Epoch 40/60 12/12 [==============================] - 64s 5s/step - loss: 1.0188 - accuracy: 0.6501 - val_loss: 1.5553 - val_accuracy: 0.5405 Epoch 00040: val_accuracy did not improve from 0.57658 Epoch 41/60 12/12 [==============================] - 64s 5s/step - loss: 0.9524 - accuracy: 0.6731 - val_loss: 1.6678 - val_accuracy: 0.5225 Epoch 00041: val_accuracy did not improve from 0.57658 Epoch 42/60 12/12 [==============================] - 64s 5s/step - loss: 0.9490 - accuracy: 0.6681 - val_loss: 1.5572 - val_accuracy: 0.5556 Epoch 00042: val_accuracy did not improve from 0.57658 Epoch 43/60 12/12 [==============================] - 64s 5s/step - loss: 0.8835 - accuracy: 0.6932 - val_loss: 1.5831 - val_accuracy: 0.5225 Epoch 00043: val_accuracy did not improve from 0.57658 Epoch 44/60 12/12 [==============================] - 74s 6s/step - loss: 0.9022 - accuracy: 0.6918 - val_loss: 1.7392 - val_accuracy: 0.5195 Epoch 00044: val_accuracy did not improve from 0.57658 Epoch 45/60 12/12 [==============================] - 67s 6s/step - loss: 0.8978 - accuracy: 0.6825 - val_loss: 1.4537 - val_accuracy: 0.5135 Epoch 00045: val_accuracy did not improve from 0.57658 Epoch 46/60 12/12 [==============================] - 65s 5s/step - loss: 0.8585 - accuracy: 0.7045 - val_loss: 1.4191 - val_accuracy: 0.5646 Epoch 00046: val_accuracy did not improve from 0.57658 Epoch 47/60 12/12 [==============================] - 66s 5s/step - loss: 0.8384 - accuracy: 0.6982 - val_loss: 1.4967 - val_accuracy: 0.5766 Epoch 00047: val_accuracy did not improve from 0.57658 Epoch 48/60 12/12 [==============================] - 65s 5s/step - loss: 0.8304 - accuracy: 0.6972 - val_loss: 1.4560 - val_accuracy: 0.5916 Epoch 00048: val_accuracy improved from 0.57658 to 0.59159, saving model to best_model.h5 Epoch 49/60 12/12 [==============================] - 65s 5s/step - loss: 0.8300 - accuracy: 0.7055 - val_loss: 1.5965 - val_accuracy: 0.5255 Epoch 00049: val_accuracy did not improve from 0.59159 Epoch 50/60 12/12 [==============================] - 73s 6s/step - loss: 0.7742 - accuracy: 0.7269 - val_loss: 1.6363 - val_accuracy: 0.5435 Epoch 00050: val_accuracy did not improve from 0.59159 Epoch 51/60 12/12 [==============================] - 73s 6s/step - loss: 0.7744 - accuracy: 0.7223 - val_loss: 1.7010 - val_accuracy: 0.5165 Epoch 00051: val_accuracy did not improve from 0.59159 Epoch 52/60 12/12 [==============================] - 70s 6s/step - loss: 0.7863 - accuracy: 0.7276 - val_loss: 1.6341 - val_accuracy: 0.5736 Epoch 00052: val_accuracy did not improve from 0.59159 Epoch 53/60 12/12 [==============================] - 68s 6s/step - loss: 0.7336 - accuracy: 0.7350 - val_loss: 1.6644 - val_accuracy: 0.5495 Epoch 00053: val_accuracy did not improve from 0.59159 Epoch 00053: early stopping
The accuracy of this model is 73% on training set. This model unfortunately does not have as good training accuracy, as compared to model1 and it appears to be underfitting on the training dataset. That means we need to increase the complexity of the model in our next attempt.
Model 3: CNN with Dropout after Convolution and having two Dense layers with 512 & 256 Units respectively
learningrate = 0.001
kernelsize = 5
model_ln4 = LNmodel(in_shape, [8,16], [256,128], kernelsize, 12, lr,
stride=1, pool_size=2, leakyrelu_slope=0.1, dropc=0.30,
dropd=0.6, bnorm=False)
model_ln4.summary()
Model: "model_3" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_4 (InputLayer) [(None, 128, 128, 3)] 0 _________________________________________________________________ conv2d_16 (Conv2D) (None, 128, 128, 8) 608 _________________________________________________________________ leaky_re_lu_12 (LeakyReLU) (None, 128, 128, 8) 0 _________________________________________________________________ dropout_24 (Dropout) (None, 128, 128, 8) 0 _________________________________________________________________ max_pooling2d_16 (MaxPooling (None, 64, 64, 8) 0 _________________________________________________________________ conv2d_17 (Conv2D) (None, 64, 64, 16) 3216 _________________________________________________________________ leaky_re_lu_13 (LeakyReLU) (None, 64, 64, 16) 0 _________________________________________________________________ dropout_25 (Dropout) (None, 64, 64, 16) 0 _________________________________________________________________ max_pooling2d_17 (MaxPooling (None, 32, 32, 16) 0 _________________________________________________________________ flatten_5 (Flatten) (None, 16384) 0 _________________________________________________________________ dense_13 (Dense) (None, 256) 4194560 _________________________________________________________________ leaky_re_lu_14 (LeakyReLU) (None, 256) 0 _________________________________________________________________ dropout_26 (Dropout) (None, 256) 0 _________________________________________________________________ dense_14 (Dense) (None, 128) 32896 _________________________________________________________________ leaky_re_lu_15 (LeakyReLU) (None, 128) 0 _________________________________________________________________ dropout_27 (Dropout) (None, 128) 0 _________________________________________________________________ dense_15 (Dense) (None, 12) 1548 ================================================================= Total params: 4,232,828 Trainable params: 4,232,828 Non-trainable params: 0 _________________________________________________________________
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=20)
mc = ModelCheckpoint('best_model.h5', monitor='val_accuracy', mode='max', verbose=1, save_best_only=True)
history_model_ln4 = model_ln4.fit(x_train, y_train_c,
validation_split=0.1,
verbose=1, batch_size=512,
shuffle=True, epochs=35,callbacks=[es,mc])
Epoch 1/35 6/6 [==============================] - 39s 6s/step - loss: 2.8455 - accuracy: 0.1066 - val_loss: 2.4646 - val_accuracy: 0.1622 Epoch 00001: val_accuracy improved from -inf to 0.16216, saving model to best_model.h5 Epoch 2/35 6/6 [==============================] - 40s 7s/step - loss: 2.4522 - accuracy: 0.1186 - val_loss: 2.4408 - val_accuracy: 0.1502 Epoch 00002: val_accuracy did not improve from 0.16216 Epoch 3/35 6/6 [==============================] - 44s 7s/step - loss: 2.4018 - accuracy: 0.1531 - val_loss: 2.3508 - val_accuracy: 0.1712 Epoch 00003: val_accuracy improved from 0.16216 to 0.17117, saving model to best_model.h5 Epoch 4/35 6/6 [==============================] - 46s 8s/step - loss: 2.2906 - accuracy: 0.2146 - val_loss: 2.2475 - val_accuracy: 0.2282 Epoch 00004: val_accuracy improved from 0.17117 to 0.22823, saving model to best_model.h5 Epoch 5/35 6/6 [==============================] - 44s 7s/step - loss: 2.1807 - accuracy: 0.2587 - val_loss: 2.1200 - val_accuracy: 0.2673 Epoch 00005: val_accuracy improved from 0.22823 to 0.26727, saving model to best_model.h5 Epoch 6/35 6/6 [==============================] - 45s 7s/step - loss: 2.0545 - accuracy: 0.2938 - val_loss: 2.0341 - val_accuracy: 0.3213 Epoch 00006: val_accuracy improved from 0.26727 to 0.32132, saving model to best_model.h5 Epoch 7/35 6/6 [==============================] - 44s 7s/step - loss: 1.9556 - accuracy: 0.3392 - val_loss: 1.9396 - val_accuracy: 0.3483 Epoch 00007: val_accuracy improved from 0.32132 to 0.34835, saving model to best_model.h5 Epoch 8/35 6/6 [==============================] - 44s 7s/step - loss: 1.8202 - accuracy: 0.3827 - val_loss: 1.8794 - val_accuracy: 0.3964 Epoch 00008: val_accuracy improved from 0.34835 to 0.39640, saving model to best_model.h5 Epoch 9/35 6/6 [==============================] - 45s 7s/step - loss: 1.6931 - accuracy: 0.4338 - val_loss: 1.7901 - val_accuracy: 0.4685 Epoch 00009: val_accuracy improved from 0.39640 to 0.46847, saving model to best_model.h5 Epoch 10/35 6/6 [==============================] - 45s 7s/step - loss: 1.5786 - accuracy: 0.4612 - val_loss: 1.6831 - val_accuracy: 0.4414 Epoch 00010: val_accuracy did not improve from 0.46847 Epoch 11/35 6/6 [==============================] - 46s 8s/step - loss: 1.4874 - accuracy: 0.4896 - val_loss: 1.7088 - val_accuracy: 0.4715 Epoch 00011: val_accuracy improved from 0.46847 to 0.47147, saving model to best_model.h5 Epoch 12/35 6/6 [==============================] - 47s 8s/step - loss: 1.4650 - accuracy: 0.5030 - val_loss: 1.6266 - val_accuracy: 0.5015 Epoch 00012: val_accuracy improved from 0.47147 to 0.50150, saving model to best_model.h5 Epoch 13/35 6/6 [==============================] - 46s 8s/step - loss: 1.3918 - accuracy: 0.5301 - val_loss: 1.6613 - val_accuracy: 0.4925 Epoch 00013: val_accuracy did not improve from 0.50150 Epoch 14/35 6/6 [==============================] - 49s 8s/step - loss: 1.3182 - accuracy: 0.5481 - val_loss: 1.4952 - val_accuracy: 0.5255 Epoch 00014: val_accuracy improved from 0.50150 to 0.52553, saving model to best_model.h5 Epoch 15/35 6/6 [==============================] - 47s 8s/step - loss: 1.2742 - accuracy: 0.5699 - val_loss: 1.5026 - val_accuracy: 0.5285 Epoch 00015: val_accuracy improved from 0.52553 to 0.52853, saving model to best_model.h5 Epoch 16/35 6/6 [==============================] - 48s 8s/step - loss: 1.2246 - accuracy: 0.5882 - val_loss: 1.4603 - val_accuracy: 0.5465 Epoch 00016: val_accuracy improved from 0.52853 to 0.54655, saving model to best_model.h5 Epoch 17/35 6/6 [==============================] - 46s 7s/step - loss: 1.1713 - accuracy: 0.5943 - val_loss: 1.4438 - val_accuracy: 0.5405 Epoch 00017: val_accuracy did not improve from 0.54655 Epoch 18/35 6/6 [==============================] - 49s 8s/step - loss: 1.1025 - accuracy: 0.6340 - val_loss: 1.4222 - val_accuracy: 0.5556 Epoch 00018: val_accuracy improved from 0.54655 to 0.55556, saving model to best_model.h5 Epoch 19/35 6/6 [==============================] - 54s 9s/step - loss: 1.0571 - accuracy: 0.6327 - val_loss: 1.3859 - val_accuracy: 0.5706 Epoch 00019: val_accuracy improved from 0.55556 to 0.57057, saving model to best_model.h5 Epoch 20/35 6/6 [==============================] - 50s 8s/step - loss: 1.0063 - accuracy: 0.6678 - val_loss: 1.4003 - val_accuracy: 0.5586 Epoch 00020: val_accuracy did not improve from 0.57057 Epoch 21/35 6/6 [==============================] - 51s 8s/step - loss: 0.9760 - accuracy: 0.6668 - val_loss: 1.3520 - val_accuracy: 0.5736 Epoch 00021: val_accuracy improved from 0.57057 to 0.57357, saving model to best_model.h5 Epoch 22/35 6/6 [==============================] - 51s 8s/step - loss: 0.9209 - accuracy: 0.6832 - val_loss: 1.3631 - val_accuracy: 0.5856 Epoch 00022: val_accuracy improved from 0.57357 to 0.58559, saving model to best_model.h5 Epoch 23/35 6/6 [==============================] - 49s 8s/step - loss: 0.8820 - accuracy: 0.7066 - val_loss: 1.2732 - val_accuracy: 0.6096 Epoch 00023: val_accuracy improved from 0.58559 to 0.60961, saving model to best_model.h5 Epoch 24/35 6/6 [==============================] - 44s 7s/step - loss: 0.8419 - accuracy: 0.7116 - val_loss: 1.3446 - val_accuracy: 0.5796 Epoch 00024: val_accuracy did not improve from 0.60961 Epoch 25/35 6/6 [==============================] - 45s 8s/step - loss: 0.8254 - accuracy: 0.7152 - val_loss: 1.3260 - val_accuracy: 0.5556 Epoch 00025: val_accuracy did not improve from 0.60961 Epoch 26/35 6/6 [==============================] - 48s 8s/step - loss: 0.8033 - accuracy: 0.7276 - val_loss: 1.4105 - val_accuracy: 0.5526 Epoch 00026: val_accuracy did not improve from 0.60961 Epoch 27/35 6/6 [==============================] - 51s 8s/step - loss: 0.7596 - accuracy: 0.7436 - val_loss: 1.4209 - val_accuracy: 0.5225 Epoch 00027: val_accuracy did not improve from 0.60961 Epoch 28/35 6/6 [==============================] - 47s 7s/step - loss: 0.7585 - accuracy: 0.7363 - val_loss: 1.4746 - val_accuracy: 0.5465 Epoch 00028: val_accuracy did not improve from 0.60961 Epoch 29/35 6/6 [==============================] - 48s 8s/step - loss: 0.7392 - accuracy: 0.7490 - val_loss: 1.3255 - val_accuracy: 0.5886 Epoch 00029: val_accuracy did not improve from 0.60961 Epoch 30/35 6/6 [==============================] - 47s 8s/step - loss: 0.6766 - accuracy: 0.7814 - val_loss: 1.2801 - val_accuracy: 0.5826 Epoch 00030: val_accuracy did not improve from 0.60961 Epoch 31/35 6/6 [==============================] - 50s 8s/step - loss: 0.6531 - accuracy: 0.7871 - val_loss: 1.2755 - val_accuracy: 0.6036 Epoch 00031: val_accuracy did not improve from 0.60961 Epoch 32/35 6/6 [==============================] - 57s 9s/step - loss: 0.6037 - accuracy: 0.8028 - val_loss: 1.3478 - val_accuracy: 0.5646 Epoch 00032: val_accuracy did not improve from 0.60961 Epoch 33/35 6/6 [==============================] - 51s 9s/step - loss: 0.5601 - accuracy: 0.8155 - val_loss: 1.4222 - val_accuracy: 0.5556 Epoch 00033: val_accuracy did not improve from 0.60961 Epoch 34/35 6/6 [==============================] - 54s 9s/step - loss: 0.5762 - accuracy: 0.8185 - val_loss: 1.4481 - val_accuracy: 0.5586 Epoch 00034: val_accuracy did not improve from 0.60961 Epoch 35/35 6/6 [==============================] - 54s 9s/step - loss: 0.5336 - accuracy: 0.8229 - val_loss: 1.4031 - val_accuracy: 0.5706 Epoch 00035: val_accuracy did not improve from 0.60961
*The accuracy on the training set is a bit better than model1. This model has the best training accuracy.
CNN Model 1
print(history.history.keys())
# summarize history for accuracy
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])
CNN Model 2
print(history_model_ln3.history.keys())
# summarize history for accuracy
plt.plot(history_model_ln3.history['accuracy'])
plt.plot(history_model_ln3.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])
CNN Model 3
print(history_model_ln4.history.keys())
# summarize history for accuracy
plt.plot(history_model_ln4.history['accuracy'])
plt.plot(history_model_ln4.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])
CNN Model 1
model.evaluate(x_test,y_test_c)
45/45 [==============================] - 18s 395ms/step - loss: 0.6089 - accuracy: 0.8147
[0.6089368462562561, 0.8147368431091309]
CNN Model 2
model_ln3.evaluate(x_test,y_test_c)
45/45 [==============================] - 6s 135ms/step - loss: 1.4218 - accuracy: 0.6211
[1.4217822551727295, 0.621052622795105]
CNN Model 3
model_ln4.evaluate(x_test,y_test_c)
45/45 [==============================] - 5s 103ms/step - loss: 1.1366 - accuracy: 0.6519
[1.1366232633590698, 0.6519297957420349]
Even though Model 3 has more accuracy on training set but its accuracy on testing data is not good. It is overfitting. Model1 is coming out as the best model. It is an imbalanced dataset.So we are more concerned with Precision and Recall. Since these two metrics are both quite important in this scenario, we will also check the F1 score to try to achieve a good balance between Precision and Recall.
As we can see, Model 1 and Model 3 seem to be generalizing well because they both have a good Holdout set Accuracy. Let us compute the confusion matrix for these two models to understand the distribution of True Positives, False Positives, False Negatives and True Negatives.
CNN Model 1
# Test Prediction
y_test_pred_m1 = model.predict(x_test)
y_test_pred_classes_m1 = np.argmax(y_test_pred_m1, axis=1)
y_test_pred_prob_m1 = np.max(y_test_pred_m1, axis=1)
y_test_labels=np.argmax(y_test_c, axis=1)
y_test_labels[1]
1
# Test Accuracy
import seaborn as sns
from sklearn.metrics import accuracy_score, confusion_matrix
accuracy_score(np.array(y_test_labels), y_test_pred_classes_m1)
0.8147368421052632
cf_matrix = confusion_matrix(np.array(y_test_labels), y_test_pred_classes_m1)
# Confusion matrix normalized per category true value
cf_matrix_n1 = cf_matrix/np.sum(cf_matrix, axis=1)
plt.figure(figsize=(12,10))
sns.heatmap(cf_matrix_n1, xticklabels=CATEGORIES, yticklabels=CATEGORIES, annot=True)
<AxesSubplot:>
CNN Model 3
#Test Prediction
y_test_pred_ln4 = model_ln4.predict(x_test)
y_test_pred_classes_ln4 = np.argmax(y_test_pred_ln4, axis=1)
y_test_pred_prob_ln4 = np.max(y_test_pred_ln4, axis=1)
import seaborn as sns
from sklearn.metrics import accuracy_score, confusion_matrix
accuracy_score(np.array(y_test_labels), y_test_pred_classes_ln4)
0.6519298245614035
cf_matrix = confusion_matrix(np.array(y_test_labels), y_test_pred_classes_ln4)
# Confusion matrix normalized per category true value
cf_matrix_n1 = cf_matrix/np.sum(cf_matrix, axis=1)
plt.figure(figsize=(12,10))
sns.heatmap(cf_matrix_n1, xticklabels=CATEGORIES, yticklabels=CATEGORIES, annot=True)
<AxesSubplot:>
The above two confusion matrices show that the models seem to be working well. Let's calculate the F1 score (the harmonic mean of precision and recall), which is used as an evaluation metric for imbalanced datasets.
Classification Report for each class
Precision: precision is the fraction of relevant instances among the retrieved instances.
Recall: recall is the fraction of relevant instances that were retrieved.
F-beta score: The F-beta score is the weighted harmonic mean of precision and recall, reaching its optimal value at 1 and its worst value at 0. The beta parameter determines the weight of recall in the combined score.
The order of printing the above metrices for each class is as follows:
CNN Model 1
from sklearn.metrics import precision_recall_fscore_support
p=precision_recall_fscore_support(np.array(y_test_labels), y_test_pred_classes_m1, average=None,labels=list(np.unique(y_test_labels)))
print(" Precision is {}\n Recall is {} \n f_beta Score is {}".format(p[0],p[1],p[2]))
Precision is [0.40196078 0.86440678 0.83333333 0.84507042 0.83333333 0.84027778 0.78378378 0.77941176 0.83974359 0.875 0.93793103 0.87962963] Recall is [0.51898734 0.87179487 0.81395349 0.98360656 0.68181818 0.84615385 0.73979592 0.8030303 0.84516129 0.60869565 0.91275168 0.81896552] f_beta Score is [0.45303867 0.86808511 0.82352941 0.90909091 0.75 0.84320557 0.76115486 0.79104478 0.84244373 0.71794872 0.92517007 0.84821429]
CNN Model 3
from sklearn.metrics import precision_recall_fscore_support
p=precision_recall_fscore_support(np.array(y_test_labels), y_test_pred_classes_ln4, average=None,labels=list(np.unique(y_test_labels)))
print(" Precision is {}\n Recall is {} \n f_beta Score is {}".format(p[0],p[1],p[2]))
Precision is [0.40909091 0.74814815 0.64356436 0.85026738 0.24761905 0.53684211 0.58203125 0.74468085 0.7398374 0.8 0.92481203 0.81481481] Recall is [0.11392405 0.86324786 0.75581395 0.86885246 0.78787879 0.35664336 0.76020408 0.53030303 0.58709677 0.4057971 0.82550336 0.56896552] f_beta Score is [0.17821782 0.8015873 0.69518717 0.85945946 0.37681159 0.42857143 0.65929204 0.61946903 0.65467626 0.53846154 0.87234043 0.67005076]
Model 1 (Best) Observation
As we see from the precision for each class, the Common wheat classifier has the highest precision. But here, we are more concerned about the case where a seedling is wrongly classified as belonging to the weed category (False Negative).
18% of Small flowered crainesbill, 17% belonging to Shepherd's Purse and 11% of sugar beet are not identified correctly, and the model predicts that they are weeds* - which shows that our model does not do well in identifying Small Flowered crainesbill, Shepherd's Purse and sugar beet. But it is works well for the other seedlings, where it does not classify them as weeds
Model 1
from sklearn.metrics import f1_score
f1_score(np.array(y_test_labels), y_test_pred_classes_m1, average='weighted')
0.8157033821513757
Model 3
from sklearn.metrics import f1_score
f1_score(np.array(y_test_labels), y_test_pred_classes_ln4, average='weighted')
0.6508960363259666
Model 1 CNN with droupout has a better F1 score.
There is still scope for improvement in the test accuracy and F1 score of the CNN model chosen here. Different architectures can be built and hyperparamter tuning can be performed to obtain a better plant seedlings classifications.
Some of the techniques which could be used for further improving the performance of the model are